Skip to content

PR-B5 (ADR 0008 Phase B): TypeScript SDK — @kakeya/runtime - #48

Merged
FluffyAIcode merged 4 commits into
mainfrom
AgentMemory/v030-pr-b5-typescript-sdk-8e7f
Jun 1, 2026
Merged

PR-B5 (ADR 0008 Phase B): TypeScript SDK — @kakeya/runtime#48
FluffyAIcode merged 4 commits into
mainfrom
AgentMemory/v030-pr-b5-typescript-sdk-8e7f

Conversation

@FluffyAIcode

Copy link
Copy Markdown
Owner

⚠️ Stacked PR

Depends on PR #47 (PR-B4). Diff against main shows both PRs;
once #47 merges, this auto-rebases to a clean PR-B5-only diff.

What

Final PR of ADR 0008 Phase B. Ships sdks/typescript/
@kakeya/runtime package matching ADR §3.2 exactly. Public API
mirrors the Python SDK at sdks/python/kakeya/* symbol-for-symbol
(modulo TS naming convention) so cross-language users see one
shape:

import { Client } from "@kakeya/runtime";

const client = new Client("localhost:50051");
try {
  const session = await client.createSession({ eosTokenIds: [151645] });
  try {
    await session.append([10, 20, 30]);
    for await (const tokenId of session.generate({ maxTokens: 64 })) {
      console.log(tokenId);
    }
  } finally {
    await session.close();
  }
} finally {
  client.close();
}

Targets

Runtime Supported
Node.js 20+ ✅ (engines.node enforced)
Electron 30+ ✅ (API-compatible with Node)
Bun 1.1+ ✅ (API-compatible with Node)
Browser ❌ (gRPC-Web / WebSocket out of scope; ADR §8 OQ-1)

Surface

File Role
src/index.ts Public exports
src/client.ts (~110 lines) Client, DEFAULT_ADDRESS, ClientOptions
src/session.ts (~285 lines) Session, SessionInfo, GenerateOptions, GenerateResult
src/errors.ts (~130 lines) KakeyaError base + 7 typed subclasses + wrapGrpcError
src/proto_gen/kakeya/v1/runtime.ts Generated by protoc-gen-ts_proto; CI's proto-stub-drift extended to enforce

Error mapping (mirrors Python exactly):

gRPC StatusCode SDK exception
NOT_FOUND SessionNotFoundError
INVALID_ARGUMENT InvalidArgumentError
FAILED_PRECONDITION InvariantViolationError
RESOURCE_EXHAUSTED ResourceExhaustedError
UNIMPLEMENTED UnimplementedError
CANCELLED RpcCancelledError
(other) KakeyaError (base, with .rpcCode)
(client-side, never on wire) SessionClosedError

Wire transport

@grpc/grpc-js (Node-native HTTP/2). Generated stubs use
protoc-gen-ts_proto with these options (recorded in
scripts/regenerate_proto_stubs.sh):

esModuleInterop=true        match tsconfig
forceLong=string            uint64 → string (JS Number can't hold full range)
outputServices=grpc-js      generate @grpc/grpc-js style Client + Service
useOptionals=messages       proto3 optional + message fields use TS optional
stringEnums=false           enums emit as integer constants (matches wire)
removeEnumPrefix=true       strip GENERATE_DONE_ prefix from enum members

forceLong=string is the safe default for uint64 fields
(historyLength, kvLiveBytes, droppedTokenCount, seed):
values cross the wire as decimal strings; the SDK converts to
Number at the public-API boundary. seed parameter accepts
number | string and stringifies internally for ergonomics.

Tests (54 vitest cases, 100% coverage)

File Tests Focus
test/client.test.ts 14 Construction, properties, idempotent close, all createSession arg paths, end-to-end RESOURCE_EXHAUSTED and NOT_FOUND via real gRPC channel
test/session.test.ts 20 All 5 RPC methods, async iterator for generate, HistoryTruncated capture, sampling param forwarding, error mappings, local-close vs runtime-eviction split, lastResult reset between calls
test/errors.test.ts 20 Every grpc.status → typed subclass, unknown-status fallback, details ?? message ?? "" fallback chain
test/server_fixture.ts (helper) In-Node @grpc/grpc-js server with parameterized canned-response handlers. Real wire format, real gRPC channel — no mocks of the SUT
npm test
54 passed | 100 % statements | 100 % branches | 100 % functions | 100 % lines

Tooling additions

scripts/regenerate_proto_stubs.sh (extended)

Single canonical regen command now produces both Python and TS
stubs. Behavior:

  • Python stubs → inference_engine/server/proto_gen/ (unchanged)
  • TS stubs → sdks/typescript/src/proto_gen/ (new)

Skips TS generation with a clear warning if ts-proto isn't
installed (avoids breaking contributors who only have Python tools).

.github/workflows/ci.yaml (extended)

Two changes:

  1. proto-stub-drift job now also drift-checks TS stubs.
    Installs Node 20 + ts-proto before regen; git diff --exit-code covers both inference_engine/server/proto_gen/
    and sdks/typescript/src/proto_gen/.

  2. typescript-sdk-tests job (new). Runs:

    • npm install in sdks/typescript/
    • npm run typecheck (strict TS)
    • npm test (vitest with 100% coverage thresholds)

Per ADR 0008 §9

Linux-only-path carve-out: zero MLX runtime code; SDK is pure TS
over @grpc/grpc-js. The Linux CI gate is the binding gate. Mac
M4 reviewers can run npm test in sdks/typescript/ to verify
on Apple Silicon, but no §9 mandatory report applies.

Reviewer checklist

  • Public API matches Python SDK symbol-for-symbol (modulo case): Client, Session, SessionInfo, 7 typed errors, DEFAULT_ADDRESS.
  • No tokenization in SDK core (grep -ri "applyChatTemplate\|tokenizer" sdks/typescript/src/ returns no production hits).
  • Generated stubs reproducible: CI's proto-stub-drift job confirms on every push.
  • Real gRPC channel in tests (no mocks of @grpc/grpc-js itself).
  • forceLong=string rationale documented in the regen script (uint64 safety).
  • engines.node>=20 declared in package.json.
  • No browser-runtime assumptions in code (Node-only @grpc/grpc-js).

Phase B is complete after this PR merges

PR-A1 (#40)  proto/runtime.proto + buf lint                  ✅ merged
PR-A2 (#41)  SessionStore + INV-1/2                          ✅ merged
PR-A3 (#42)  remove ADR 0007 dead code                       ✅ merged
PR-A3b(#43)  session-slab ownership + CacheInspector         ✅ merged
PR-B1 (#44)  gRPC Create/Close/GetSessionInfo                ✅ merged
PR-B2 (#45)  AppendTokens + byte-exact prefill-incremental   ✅ merged
PR-B3 (#46)  Generate server-streaming + greedy              ⏳ ready
PR-B4 (#47)  Python SDK (kakeya.Client + Session)            ⏳ ready
PR-B5 (this) TypeScript SDK (@kakeya/runtime)                ⏳ this PR

What's next (post-Phase-B)

  • Phase C / PR-D1: Refactor the deprecated HTTP shim onto the new SessionStore (today the HTTP shim still uses PooledVerifier directly).
  • Phase E / PR-E1: tests/integration/ directory + the long-promised test_inv3_session_determinism_gate.py running against the real Qwen3 verifier on Mac M4.
  • v0.4 deferreds: multi-tenant authentication, speculative-decoding integration into Generate, browser-targeted SDK transport (revisit OQ-1).
Open in Web Open in Cursor 

cursoragent and others added 4 commits June 1, 2026 14:06
…ion-aware decoding

First Phase-B PR with mandatory Mac M4 integration test report
under \u00a79 (Linux CI is necessary but not sufficient).

Design choice: PR-B3 ships **greedy decoding only**. Speculative-
decoding integration (DLM proposer + AR verifier rejection sampling)
is reserved for a later PR. The wire contract is algorithm-agnostic
(GenerateResponse oneof on token_id / done / truncated), so the
upgrade path lands without breaking clients.

Three deliverables:

  1. inference_engine/session/generator.py (new, ~240 lines)
     - VerifierProtocol-driven GenerationCoordinator.
     - Yields TokenEvent / HistoryTruncatedEvent / DoneEvent.
     - Strict v0.3 validation: temperature/top_p/top_k must be in
       greedy no-op defaults; max_tokens must be >= 1; session must
       have prior AppendTokens. Per \u00a72.10 'no graceful degradation',
       any deviation raises ValueError -> INVALID_ARGUMENT.
     - HistoryTruncated emitted at most once per call, BEFORE any
       TokenEvent, when the cache is in sink+window-truncated state
       (matches the runtime.proto contract exactly).
     - INV-1 / INV-2 enforced via SessionStore at every step.
     - INV-3 byte-exact under greedy: same (session_id, history)
       -> bit-identical token stream. Tested with FakeVerifier on
       Linux and asserted in TestDeterminism.

  2. inference_engine/server/grpc_app.py (modified, +119 lines)
     - RuntimeServiceServicer.__init__ takes optional
       generation_coordinator: GenerationCoordinator. None = PR-B2
       UNIMPLEMENTED default preserved (regression-tested).
     - Generate RPC implements server-streaming with the four typed
       error mappings:
         SessionNotFoundError  -> NOT_FOUND
         ValueError            -> INVALID_ARGUMENT
         InvariantViolation    -> FAILED_PRECONDITION
       Plus the success path streams TokenEvent -> token_id,
       HistoryTruncatedEvent -> truncated, DoneEvent -> done.
     - Cancellation: polls context.cancelled() between events; on
       True, emits a final GenerateDone(STOP_REASON_CANCELLED) and
       returns. Cancellation latency is bounded by one generation
       step (the in-flight forward pass finishes before the next
       poll).
     - create_grpc_server factory plumbed with the new keyword.

  3. tests/inference_engine/session/test_generator.py (new, 31 tests)
     - TestGreedyHappyPath: 4 tests on token-then-done emission,
       max_tokens cap, single Done event, total_seconds reporting.
     - TestGreedyAdvancesVerifier: 4 tests confirming verifier
       state mirrored onto session after every step.
     - TestEos: 3 tests on EOS detection + STOP_REASON_EOS.
     - TestHistoryTruncated: 3 tests on at-start emission + at-most-
       once contract.
     - TestValidation: 9 tests on max_tokens, sampling params, and
       no-AppendTokens-prior rejection.
     - TestInvariants: 2 tests on INV-1 / INV-2 propagation.
     - TestDeterminism: 1 test on byte-exact greedy output across
       parallel sessions.
     - TestConstructorAndEventDataclasses: 4 tests on frozen dataclass
       contract and constructor.

  + tests/inference_engine/server/test_grpc_app.py (extended, +11 tests)
     - Generate streams tokens then done.
     - EOS triggers STOP_REASON_EOS.
     - HistoryTruncated frame emitted before tokens.
     - Unknown session -> NOT_FOUND.
     - No AppendTokens prior -> INVALID_ARGUMENT.
     - max_tokens=0 -> INVALID_ARGUMENT.
     - temperature=0.7 -> INVALID_ARGUMENT.
     - seed accepted on the wire.
     - InvariantViolation during generation -> FAILED_PRECONDITION.
     - Cancellation -> STOP_REASON_CANCELLED Done frame (direct
       Servicer invocation with FakeContext; cancellation latency
       bounded as documented).
     - Factory accepts generation_coordinator keyword.

Mac M4 reviewer aids:

  scripts/smoke_grpc_generator.py
    10-scenario smoke walking Generate scenarios with Append +
    Generate coordinators wired against FakeVerifier.

  scripts/review_pr_b3_on_mac.sh
    One-shot Mac M4 reviewer producing 5 JSON artifacts:
      pr-b3-mac-generator-tests-<unix>.json (31 tests, 100% on
      generator.py)
      pr-b3-mac-grpc-tests-<unix>.json (39 tests after PR-B3
      additions, 100% on grpc_app.py)
      pr-b3-mac-grpc-runtime-smoke-<unix>.json (PR-B1 regression)
      pr-b3-mac-grpc-appender-smoke-<unix>.json (PR-B2 regression)
      pr-b3-mac-grpc-generator-smoke-<unix>.json (PR-B3 new)

Local verification (Linux VM, py3.12):
  Linux CI gate: 629 passed (was 587 + 31 generator + 11 grpc
                 generate). Coverage 100.00% on 1521 stmts (was
                 1428 + 63 generator + 30 grpc_app additions
                 - 0 deletions).

Per ADR 0008 \u00a79: this PR introduces a NEW MLX-reachable runtime
path (verifier.forward_block + commit_or_truncate driven by the
coordinator's per-token loop). Even though FakeVerifier covers the
Linux-side dispatch logic exhaustively, the v0.3 GA contract
requires a Mac M4 integration test report on the PR branch before
merge. The carve-out paragraph does NOT apply here. PR is opened as
Draft pending the report.

Mac M4 verification command for the PR branch:
  bash scripts/review_pr_b3_on_mac.sh

The script writes 5 JSON artifacts under results/platform-tests/;
the user commits them back to this branch and the PR description is
updated to quote the smoke summary line.

Co-authored-by: FluffyAIcode <FluffyAIcode@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…ssion

Stacks on PR-B3 (#46). When this PR is merged, PR-B3 lands along
with it.

Public API matches the ADR 0008 \u00a73.1 example exactly:

    from kakeya import Client

    with Client('localhost:50051') as client:
        with client.create_session(eos_token_ids=[151645]) as session:
            session.append([10, 20, 30])
            for token_id in session.generate(max_tokens=64):
                print(token_id)

Surface (sdks/python/kakeya/):

  client.py (Client)
    - .address / .closed properties
    - .create_session(eos_token_ids, client_label) -> Session
    - .close() / context manager
    - Wraps grpc.insecure_channel + RuntimeServiceStub (sync)

  session.py (Session, SessionInfo)
    - .session_id / .closed
    - .last_stop_reason / .last_generated_token_count /
      .last_prefill_duration_seconds / .last_total_duration_seconds /
      .last_history_truncated_dropped (populated by .generate)
    - .append(token_ids) -> int
    - .generate(max_tokens, seed, temperature, top_p, top_k) -> Iterator[int]
    - .info() -> SessionInfo
    - .close() / context manager
    - SessionClosedError on local-side reuse after close()

  errors.py (typed exception hierarchy)
    - KakeyaError (base, carries .rpc_code)
    - SessionNotFoundError       <- gRPC NOT_FOUND
    - InvalidArgumentError       <- gRPC INVALID_ARGUMENT
    - InvariantViolationError    <- gRPC FAILED_PRECONDITION
    - ResourceExhaustedError     <- gRPC RESOURCE_EXHAUSTED
    - UnimplementedError         <- gRPC UNIMPLEMENTED
    - RpcCancelledError          <- gRPC CANCELLED
    - SessionClosedError         <- client-side, never crosses the wire

Sync vs async: PR-B4 ships sync only (grpc.insecure_channel). The
ADR \u00a73.1 example is sync, and the v0.3 target audience (REPL,
scripts, agent harnesses on a single Mac) prefers sync. An async
API can be added in a later PR without breaking this surface; the
runtime is identical (grpc.aio.server is wire-compatible with sync
clients).

Tokenization: per ADR \u00a72.4 / \u00a73.4, the SDK ships ZERO chat-template
logic. The runtime treats token ids as opaque integers; rendering
messages to tokens lives in sdks/python/examples/ (a future PR).

Tests (tests/sdk/python/):

  test_errors.py       (20 tests)
    Every gRPC StatusCode -> typed-exception mapping verified.
    Plus the catch-all path for unknown codes, the .rpc_code carry,
    and the SessionClosedError no-rpc-code variant.

  test_client.py       (13 tests)
    Construction, address property, closed flag, idempotent close,
    context manager, create_session with all argument combinations,
    RESOURCE_EXHAUSTED -> ResourceExhaustedError end-to-end.

  test_session.py      (33 tests)
    All 5 RPC methods, all error mappings, last_* metadata fields,
    HistoryTruncated -> .last_history_truncated_dropped, sampling
    param validation (greedy-only contract surfaces as
    InvalidArgumentError), context manager, double-close idempotency,
    SessionInfo dataclass standalone construction + repr.

All 66 tests run against a real grpc.aio runtime spun up in a
background thread (see tests/sdk/python/conftest.py). The
fixture creates the gRPC server INSIDE the worker thread's loop
(not the main thread) to avoid 'Future attached to a different
loop' from grpcio's internals; shutdown uses
server.wait_for_termination() so the loop can exit cleanly without
a cancellation tantrum.

CI changes (.github/workflows/ci.yaml):
  - PYTHONPATH gains sdks/python so 'import kakeya' resolves
    without pip-install (consistent with the rest of the repo's
    PYTHONPATH-based layout).
  - tests/sdk/python/ added to pytest paths.
  - --cov=kakeya added to the coverage gate.
  - package-import-smoke imports kakeya, kakeya.client,
    kakeya.session, kakeya.errors (regression catches an
    accidentally-broken __init__).

Mac M4 reviewer aid (\u00a79 carve-out applies — Linux-only path):
  scripts/review_pr_b4_on_mac.sh
    Runs SDK pytest + 100% coverage gate, plus all three prior
    PR-B1/B2/B3 smokes for regression. Produces 4 JSON artifacts
    under results/platform-tests/.

Local verification (Linux VM, py3.12):
  Linux CI gate: 695 passed (was 629 + 66 new). Coverage 100.00%
                 on 1694 stmts (was 1521 + 173 new = sdk surface).
  Reviewer script end-to-end:
    SDK tests       : 66 tests, cov=173/173=100%
    Runtime smoke   : 10/10
    Appender smoke  : 10/10
    Generator smoke : 10/10

Next PR after merge:
  PR-B5 (\u00a76.2): TypeScript SDK under sdks/typescript/. Targets
        Node.js 20+ / Electron 30+ / Bun 1.1+. Generates stubs
        from proto/kakeya/v1/runtime.proto via protoc-gen-ts_proto.
        Linux-only path; \u00a79 carve-out continues to apply.

Co-authored-by: FluffyAIcode <FluffyAIcode@users.noreply.github.com>
Final PR of Phase B. Stacks on PR-B4 (#47); when this merges,
PR-B4 lands along with it.

Public API matches the ADR 0008 \u00a73.2 example exactly and mirrors
the Python SDK at sdks/python/kakeya/* symbol-for-symbol (modulo
TypeScript naming convention: PascalCase classes, camelCase
fields). Cross-language symmetry makes the wire contract auditable
by simple grep.

  import { Client } from '@kakeya/runtime';

  const client = new Client('localhost:50051');
  try {
    const session = await client.createSession({ eosTokenIds: [151645] });
    try {
      await session.append([10, 20, 30]);
      for await (const tokenId of session.generate({ maxTokens: 64 })) {
        console.log(tokenId);
      }
    } finally {
      await session.close();
    }
  } finally {
    client.close();
  }

Targets per ADR \u00a73.2:
  Node.js >= 20.0.0 (engines.node enforced in package.json)
  Electron 30+ and Bun 1.1+ are API-compatible (not separately
  CI-tested in v0.3 to keep the matrix small; documented support).
  Browser intentionally NOT supported (gRPC-Web / WebSocket are
  out of scope for v0.3 \u2014 see ADR 0008 \u00a78 OQ-1).

Surface (sdks/typescript/src/):

  index.ts (public exports)
  client.ts (Client + DEFAULT_ADDRESS + ClientOptions)
  session.ts (Session + SessionInfo + GenerateOptions + GenerateResult)
  errors.ts (KakeyaError base + 7 typed subclasses + wrapGrpcError)
  proto_gen/kakeya/v1/runtime.ts (generated, not hand-edited)

Wire transport: @grpc/grpc-js (Node-native HTTP/2). Generated
stubs come from protoc-gen-ts_proto with options:
  esModuleInterop=true,forceLong=string,outputServices=grpc-js,
  useOptionals=messages,stringEnums=false,removeEnumPrefix=true

forceLong=string is the safest choice for uint64 fields (history
lengths, kv_live_bytes, dropped_token_count, seed): JS Number
can't safely represent the full uint64 range, so values cross
the wire as decimal strings and the SDK converts to Number /
keeps as string at the public API boundary.

Error mapping (mirrors Python exactly):
  gRPC NOT_FOUND            -> SessionNotFoundError
  gRPC INVALID_ARGUMENT     -> InvalidArgumentError
  gRPC FAILED_PRECONDITION  -> InvariantViolationError
  gRPC RESOURCE_EXHAUSTED   -> ResourceExhaustedError
  gRPC UNIMPLEMENTED        -> UnimplementedError
  gRPC CANCELLED            -> RpcCancelledError
  (other status)            -> KakeyaError (base, with .rpcCode)
  (client-side)             -> SessionClosedError

Tests (sdks/typescript/test/, 54 vitest cases):

  client.test.ts (14 tests) \u2014 construction, properties,
    idempotent close, all createSession argument paths,
    RESOURCE_EXHAUSTED + NOT_FOUND end-to-end via real gRPC.

  session.test.ts (20 tests) \u2014 all 5 RPC methods, async
    iterator semantics for generate(), HistoryTruncated capture,
    sampling-param wire forwarding, error mappings (NOT_FOUND,
    INVALID_ARGUMENT, FAILED_PRECONDITION), local-close vs
    runtime-eviction split, lastResult reset between calls.

  errors.test.ts (20 tests) \u2014 every gRPC StatusCode -> typed
    subclass, unknown-status fallback, message fallback chain
    (details ?? message ?? '').

  test/server_fixture.ts \u2014 in-Node @grpc/grpc-js test server with
    parameterized canned-response handlers. Real wire format,
    real gRPC stream, real channel \u2014 no mocks of the SUT.

Tooling:

  package.json \u2014 npm package metadata; type=module; engines.node>=20.
  tsconfig.json \u2014 strict mode, ES2022 target, NodeNext modules.
  vitest.config.ts \u2014 100% coverage thresholds (lines, branches,
    functions, statements all enforced); proto_gen excluded since
    those are protoc's artifact, not our surface (mirrors the
    Python .coveragerc omit pattern).

  scripts/regenerate_proto_stubs.sh \u2014 extended to also produce
    TypeScript stubs via grpc_tools.protoc + protoc-gen-ts_proto.
    Single canonical regen command for all SDKs.

  .github/workflows/ci.yaml:
    - proto-stub-drift job extended: now also drift-checks TS
      stubs (regenerates and git-diff-exit-codes). Adds Node 20
      + ts-proto install before the regen step.
    - typescript-sdk-tests new job: setup-node@v4 (Node 20),
      npm install in sdks/typescript/, typecheck, vitest with
      100% coverage gate.

Local verification (Linux dev VM):
  npm test:        54 passed, 100% on lines/branches/functions/statements
  npm typecheck:   clean
  Python gate:     695 passed (unchanged, PR-B5 is TS-only)
  Regen idempotent: scripts/regenerate_proto_stubs.sh produces
                    byte-identical Python + TS stubs to committed.

Per ADR 0008 \u00a79: Linux-only path \u2014 zero MLX runtime code; SDK is
pure TypeScript over @grpc/grpc-js. \u00a79 carve-out applies.

Phase B is complete after this PR merges:
  PR-A1 (#40)  proto/runtime.proto + buf
  PR-A2 (#41)  SessionStore + INV-1/2
  PR-A3 (#42)  remove ADR 0007 dead code
  PR-A3b(#43)  session-slab ownership + CacheInspector
  PR-B1 (#44)  gRPC Create/Close/GetSessionInfo
  PR-B2 (#45)  AppendTokens + byte-exact prefill-incremental
  PR-B3 (#46)  Generate server-streaming + greedy decoding
  PR-B4 (#47)  Python SDK (kakeya.Client + Session)
  PR-B5 (this) TypeScript SDK (@kakeya/runtime)

Phase C and beyond: deprecated HTTP shim refactor (PR-D1),
integration test suite (PR-E1), v0.4 multi-tenant + speculative
decoding integration are downstream concerns.

Co-authored-by: FluffyAIcode <FluffyAIcode@users.noreply.github.com>
@FluffyAIcode
FluffyAIcode marked this pull request as ready for review June 1, 2026 15:30
@FluffyAIcode
FluffyAIcode merged commit 0642902 into main Jun 1, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants